Posons-nous et profitons de ce talk pour prendre un peu de hauteur sur l’état de l’industrie tech autour de la création d’API de persistence (CRUD). D’où venons-nous, ou allons-nous ? Pourquoi le choix entre RPC, SOAP, REST et GraphQL n’est peut-être qu’un sujet de surface qui cache un problème bien plus profond… Youtube: https://www.youtube.com/watch?v=IskE3m3VjRY

1. Choisir entre une API
RPC, SOAP, REST, GraphQL?
Et si le problème était ailleurs ?

2. François-Guillaume Ribreau
—
Architect & Head of development @Ouest-France
🌟 SaaS founder of _______ ____ MotionDynamic,
Mailpopin
🚀 Trainer @EPSI_Nantes @UnivNantes
📢 Twitter/Github: @FGRibreau

3. <quick>
<history>

4. - pattern: procedure(arguments...): [result]
- ~1980
- principle: simplest form of API interaction
- goal: run a task on another address space
- often requires an IDL (Interface Definition Language)
- well-known implementations:
- XML-RPC (XML to encode calls & HTTP as transport)
- JSON-RPC (JSON to encode calls & HTTP as transport)
- Thrift, gRPC, Avro... and *lots* of other implementation
RPC - Remote Procedure Call
client api
procedure(arguments...)
result

5. - ~successor of XML-RPC
- XML-based messaging framework
- can work with any transport protocol (just need XML)
- XML everywhere
- IDL: WSDL, a schema of every available operations
- 1998 (+18 yrs after RPC principle)
- Spec WG closed on 10 July 2009 (+11 yrs)
SOAP - Simple Object Access Protocol
client api
envelope(message(call))
envelope(message(response))

6. - Roy Fielding, 2000 (20 yrs after RPC)
- based on the uniform and predefined set of stateless
operations from HTTP protocol
- request/response
- server-side data are made available through
representations of data in simple formats (e.g. JSON, XML)
- IDL: no schema required (state-of-the-art: OpenAPI)
REST - Representational State Transfer
client api
HTTP request
HTTP response

7. More power on the client-side at request time
GET /ressources/?fields=property1,property2
GET /ressources/:ressource_id?fields=property1,property2
(e.g. Google APIs, Facebook APIs)
REST - Representational State Transfer
client api
HTTP request
HTTP response

8. GET /fql?
q=SELECT+uid2+FROM+friend+WHERE+uid1=me()&a
ccess_token=…
FQL @ Facebook (2007)
https://query.yahooapis.com/v1/public/yql?
q=select title, link from rss
where url='https://engt.co/2JyTaQP'
&format=json
&env=store://datatables.org/alltableswithkeys
YQL @ Yahoo (2008)

9. - Another RPC protocol (not a Graph db)
- 3 verbs (in the REST sense):
- fetch (implicit, read requests)
- subscription (explicit, read streams)
- mutation (writes requests)
- introduce inversion of control between client & server
- client only request needed data
GraphQL
client api
HTTP request
HTTP response

10. </history>
</quick>

11. Wow
much work
so specifications
client api

12. Current
State of the art

13. 3-tier
Database
API
Client

14. 3-tier
Database
(Tables / Views)
API
(Models)
Client

15. Validation
Database
(Schema (constraint))
API
(Models (validation))
Client
(validation)

16. Authorization
Database
(Users/roles, policies)
API
(Authorization
middleware)
Client

17. Etc… 🕰
Database
…
API
…
Client

18. Only for
data
persistence
Without
data
persistence
Mostly for
data
persistence
(+ some
MQueing)
What kind of API do you write the most?
~90% of the time
it's. just. CRUD.

19. effort =>
data
api
client
user

20. data
api
client
user
effort =>

21. Let's build a
persistence API

22. api.motiondynamic.tech
Fast highly dynamic video generation at scale 🚀

23. api
SQLHTTP
data
(PostgreSQL)
• HTTP request handling
• Authentication
• Authorization
• Request Parsing
• Request Validation
• Database Communication
• Database Response Handling (deserialization)
• Output format serialization
• HTTP Response Building
Persistence API
our job

24. api
SQLHTTP
Persistence API
TL;DR: HTTP <-> SQL mapping
… with a lot of space for potential mistakes. our job
data
(PostgreSQL)

25. Database Modelling 101
v1_0 schema authentication schema
theme schema
video schema
….
view schemastored
fn
generations
signIn signUp
themes

26. api
(PostgREST)
data
(PostgreSQL)
Persistence API
<= our job
#SSoT #DRY
v1_0
schema
client
HTTP / REST / (JSON,CSV)

27. Are we
serious?

28. Schema modelling
public private
v1_0 schema authentication schema
theme schema
video schema
….
view schemastored
fn
projects signIn signUp
Postgrest
videos
generations
startVideoGeneration

29. Read / Write requests
(read, view) GET /themes
(read, view) GET /videos
(read, view) GET /generations
(write, stored function) POST /rpc/signUp
(write, stored function) POST /rpc/signIn
(write, stored function) POST /rpc/startVideoGeneration
DBv1_0
schema
api
(PostgREST)

30. GET /themes?downloads=gte.1000&isMine=is.true
GET /themes?select=downloads::text,name,description
GET /stuff?metadata->a->>b=eq.2
GET /themes?select=id,name,generations{createdAt}
&order=name.asc&generations.order=createdAt.desc
How do you manage
projection, filtering, ordering?

31. Read / Write requests
(read, view) GET /themes
(read, view) GET /videos
(read, view) GET /generations
(write, stored function) POST /rpc/signUp
(write, stored function) POST /rpc/signIn
(write, stored function) POST /rpc/startVideoGeneration
(write) POST /generations

32. INSTEAD OF
create trigger generations_mutation
INSTEAD OF INSERT OR UPDATE OR DELETE
ON v1_0.generations
for each row execute procedure
video.mutation_generations_trigger()

33. Trigger
create or replace function video.mutation_generations_trigger() returns trigger as $$
declare
begin
if (tg_op = 'DELETE') then
-- delete generation
return new;
elsif (tg_op = 'UPDATE') then
-- update generation
return new;
elsif (tg_op = 'INSERT') then
-- create generation
return new;
end if;
end;
$$ security definer language plpgsql;
pl/pgsql ? It's 2018: pl/v8 (javascript) - pl/python - pl/java - pl/tcl

34. State-of-the-art API performance
api
data
(PostgreSQL)
client
HTTP / REST / JSON
deserialize db response
serialize http JSON response
serialize to internal representation

35. API performance 🚀
api
(PostgREST)
data
(PostgreSQL)
client
HTTP / REST / JSON
serialize result to JSON
“PostgreSQL JSON encoder is fast, it’s C fast.
For a large JSON object graph PostgreSQL JSON
generation can offer well over 12x the throughput.
It can be 2x/10x faster than ActiveRecord or even 160x
for large responses”
https://hashrocket.com/blog/posts/faster-json-generation-with-postgresql

36. Versioning
public private
v1_0 schema
v2_0 schema
authentication schema
theme schema
video schema
….
view schemastored
fn
videos signIn signUp
generations logIn signUp

37. How do you manage authentication?

38. How do you manage authorization?
CREATE ROLE authenticator NOINHERIT LOGIN;
CREATE ROLE anonymous;
CREATE ROLE authenticated_user;
GRANT anonymous, authenticated_user TO authenticator;

39. How do you manage authorization?
Row Level Security (PG 9.5+)
ALTER TABLE video.generation ENABLE ROW LEVEL SECURITY;
create policy generation_access_policy on video.generation to api
-- a user can only see its own generations, admin can see everything
using (
(request.user_role() = 'videoRequester' and user_id =
request.user_id()) or request.user_role() = 'admin')
-- only admin can change anyone generation data
with check (request.user_role() = 'admin');
3 lines of SQL
Reliable security model (closed by default)
Declarative
Expressive

40. Imperative
Programming
"how to do"
%📺'
Declarative
Programming
"what to do"
(🚀)

41. How do you manage
emails/3rd parties?
pg_notify (PG 9.2+)
http://bit.ly/2oNbaKy

42. How do you manage
emails/3rd parties?
pg_notify (PG 9.2+)
http://bit.ly/2JDCsQu

43. Rate-limiting? Headers?
Monitoring? Tracing? Custom-logic ?
Respect the separation of concerns
(that's what Service Mesh is all about btw)
data
api
client
user
proxy
(nginx/openresty | istio proxy | ...)

45. How to manage documentation?
OpenAPI (Swagger) format
automatically extracted from schema

46. How to manage
code-reviews, tests?
It’s just SQL.
SQL unit test: pgTAP
Functional tests: Supertest / Jest

47. How to manage migrations?
Full migration management
sqitch/apgdiff

48. How to build & deploy?
local env. + docker 🐳
(watch + SQL auto-reload)
Cloud (PostgreSQL + PostgREST + OpenResty)
apply migrations
rolling deploy PostgREST/OpenResty
run unit & functional tests
test migrations

49. One
more
thing

50. REST & GraphQL api
(PostgREST + SubZero)
data
(PostgreSQL)
<= our job
#SSoT #DRY
v1_0
schema
client
GraphQL ?

51. subZero ❤
GraphQL & REST (PostgREST) API for your database

52. This philosophy is
gaining traction

53. PostgraphQL - A GraphQL API created by reflection
over a PostgreSQL schema: NodeJS + PostgreSQL
Prisma - Prisma turns your database into a realtime
GraphQL API: Scala + MySQL/Postgres (upcoming: MongoDB,
ElasticSearch)
Graphql-engine - Instant GraphQL APIs on Postgres
with fine grained access control: NodeJS + PostgreSQL

54. Et si le problème était ailleurs ?
API
data
(BDD)
v1_0
schema
client
REST GraphQL Tomorrow?

55. Free plans for Redis
administration & monitoring
at redsmin.com
We are looking for Front-end Developers
twitter.com/iadvizetech
Questions?
@FGRibreau
No more server-side rendering pain,
1 url = 1 chart
image-charts.com

56. https://apple.github.io/foundationdb/transaction-manifesto.html
Performance and scalability?
We know of no practical limits to the scalability or performance of systems supporting
transactions. When the movement toward NoSQL databases began, early systems, such
as Google Bigtable, adopted minimal designs tightly focused on the scalability and
performance. Features familiar from relational databases had been aggressively shed and
the supposition was that the abandoned features were unnecessary or even harmful
for scalability and performance goals.
Those suppositions were wrong. It is becoming clear that supporting transactions is a
matter of engineering effort, not a fundamental tradeoff in the design space.
Algorithms for maintaining transactional integrity can be distributed and scale out like
many other problems.
Transactional integrity does come at a CPU cost, but in our experience that cost is less
than 10% of total system CPU. This is a small price to pay for transactional integrity and
can easily be made up elsewhere.

57. I am quite convinced that in fact computing will
become a very important science. (🐶💩)
But at the moment we are in a very
primitive state of development; we don't
know the basic principles yet and we must
learn them first.
If universities spend their time teaching the
state of the art, they will not discover
these principles and that, surely, is what
academics should be doing.
— Christopher Strachey, 1969 (49 yrs ago)
(quote by Edsger W. Dijkstra)
https://bit.ly/2pMI7aJ
“
”